Skip to content

fix(integrations): restore a rewritten file in place on rollback - #5470

Open
echarrod wants to merge 9 commits into
github:mainfrom
echarrod:fix/rollback-restore-in-place
Open

echarrod wants to merge 9 commits into
github:mainfrom
echarrod:fix/rollback-restore-in-place

Conversation

@echarrod

Copy link
Copy Markdown
Contributor

Description

Follow-up to #5402, and a prerequisite for #5117. When a built-in init or upgrade fails and the lifecycle journal rolls back, _restore_snapshots() deleted each snapshotted file and recreated it with shutil.copy2. The bytes came back, but a recreated file is a new file. On Windows it loses its access-control list and can inherit broader permissions from its directory. On POSIX it loses its owner and any hard links. Copilot raised this on #5117 (#5117 (comment)): its openai.yaml rewrite keeps the file's ACL in place, but a rollback through the journal would undo that.

The journal now records each snapshotted file's identity (device and inode), but only when the file did not change while it was being snapshotted. On rollback, if that same file is still at the path, it is opened without following links (and without blocking on a FIFO). Rollback checks that it is still the same file, then writes the original bytes back in place. Mode and times are restored through the open descriptor, so a swapped-in symlink cannot redirect them. A file that was removed, replaced, or cannot be opened is recreated exactly as before.

Limit: files inside a snapshotted directory are still deleted and recreated. Only individually snapshotted files are restored in place.

Testing

  • test_builtin_upgrade_rollback_restores_a_rewritten_file_in_place (tests/specify_cli/integrations/test_command_upgrade.py) hard-links a customised SKILL.md, then makes integration upgrade --force fail after the file has been rewritten. It checks that the original bytes and mode are back and that it is the same file. The test fails on main and passes with this change.

  • tests/specify_cli/integrations/test_lifecycle.py adds two tests. A file that the operation replaced is recreated, not written through. A file that rollback cannot open falls back to being recreated.

  • Full suite on macOS: 16801 passed, 616 skipped, 1 failed. The failure, test_version_execution.py::TestWindowsUvToolEnvironment::test_non_windows_uv_tool_dry_run_still_previews_installer, also fails on main locally: the long venv path wraps in the output. uvx ruff@0.15.0 check src tests and git diff --check are clean.

  • I could not test Windows ACLs directly. That they are kept follows from the file keeping its identity, which the tests check.

  • Tested locally with .venv/bin/specify --help (Windows PowerShell: .\.venv\Scripts\specify.exe --help)

  • Ran relevant existing tests with .venv/bin/python -m pytest tests -q (Windows PowerShell: .\.venv\Scripts\python.exe -m pytest tests -q); use affected test paths for focused checks and LC_ALL=en_US.UTF-8 on macOS

  • Tested with a sample project (if applicable)

AI Disclosure

  • I did not use AI assistance for this contribution
  • I did use AI assistance (fill in the disclosure below)

AI disclosure: Claude Code (model: claude-opus-5-5), running autonomously. The agent wrote the code, the tests and this description, and ran a /code-review pass on the change. No line-by-line human review before opening.

🤖 Generated with Claude Code

https://claude.ai/code/session_015w5Wio5NK7XbSPGaMEKqVe

echarrod and others added 2 commits November 17, 2026 22:39
Rolling back a failed operation unlinked each changed file and recreated
it from its snapshot with shutil.copy2. The new file took the restoring
user as owner and the directory's default permissions, which on Windows
replaces a restricted file's access-control list with inherited entries.

The journal now records the (st_dev, st_ino) of each regular file it
snapshots. When that same file is still at the path, rollback opens it
without truncating or following links, checks the opened file is the
snapshotted one, then truncates it, writes the snapshot bytes back and
restores mode and times with copystat. A file that was removed or
replaced by another file, a symlink or a directory, or that cannot be
opened for writing, is recreated as before.

Assisted-by: Claude Code (model: claude-opus-5-5, autonomous)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015w5Wio5NK7XbSPGaMEKqVe
Rolling back a file in place set its mode and times with copystat after
closing it, which follows a symlink swapped in at that path. Set them
through the open descriptor where the platform supports it, open with
O_NONBLOCK so a FIFO swapped in cannot block the open, and record a
snapshotted file's identity only when it did not change while the
snapshot was taken. Cover the fallback to recreating a file rollback
cannot open, and check the upgrade rewrote the file before failing.

Assisted-by: Claude Code (model: claude-opus-5-5, autonomous)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015w5Wio5NK7XbSPGaMEKqVe
@echarrod
echarrod requested a review from mnriem as a code owner November 17, 2026 22:43
Copilot AI adaptive review requested due to automatic review settings November 17, 2026 22:43

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Rollback can redirect metadata updates to an unintended file, leave extended attributes altered, and abandon remaining recovery steps after metadata permission errors.

4 open issues
What changed in this PR

Updates integration lifecycle rollback to restore individually snapshotted files in place, aiming to preserve file identity, ownership, hard links, and Windows access-control lists. This follows #5402 and supports the rollback requirements for #5117.

Changes:

  • Record regular-file identities during snapshot creation.
  • Restore matching files in place, with recreation as a fallback.
  • Add regression coverage for rewritten files, replacements, and open failures.

Validation was limited to isolated standard-library checks; project tests and Windows validation were not run.

AI-generated review for the requester by GitHub Copilot (model: unknown; autonomous).

File Description
tests/​specify_cli/​integrations/​test_lifecycle.py Tests recreation after file replacement or an in-place open failure.
tests/​specify_cli/​integrations/​test_command_upgrade.py Tests rollback preserving a rewritten file’s identity, bytes, and mode.
src/​specify_cli/​integrations/​_lifecycle.py Records snapshot identities and adds in-place file restoration.

Review effort: Max (adaptive)

@mnriem

mnriem commented Nov 17, 2026

Copy link
Copy Markdown
Collaborator

Please address Copilot feedback

@mnriem mnriem added the triage-must-have Verdict: high-value, important work for Spec Kit — do first label Nov 17, 2026
…stores

Restoring a file in place now restores its user extended attributes (where
the os module has an API for them, which in CPython means Linux), times and
mode only through the verified descriptor. Windows, which has no descriptor
form of os.utime, sets them by path while the open handle still pins that
file; any other platform without a descriptor form leaves them. A refused or
unavailable metadata change no longer aborts rollback: the bytes stay
restored, each remaining attribute, entry and directory cleanup still runs,
and the file is reported with the recovery snapshots retained.

Assisted-by: Claude Code (model: claude-opus-5-5, autonomous)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015w5Wio5NK7XbSPGaMEKqVe
Copilot AI adaptive review requested due to automatic review settings November 18, 2026 09:57
@echarrod

Copy link
Copy Markdown
Contributor Author

@mnriem I've addressed the four Copilot threads in c3f7e18.

  • In-place rollback now restores mode, times and user.* extended attributes only through the verified descriptor. Windows has no descriptor os.utime, so there the times are set by path while the open handle still pins the file. Anywhere else without descriptor support, the metadata is left alone.
  • A metadata change that is refused or unavailable no longer aborts rollback. The bytes stay restored, every other attribute, entry and directory cleanup still runs, and the file is reported in a separate warning with the recovery snapshots kept.
  • New tests in tests/specify_cli/integrations/test_lifecycle.py cover the metadata refusal, the symlink swapped in without descriptor support, a descriptor that opens a different file, and extended attributes. The extended-attribute tests run only on Linux, and I ran them in a container. A Windows-only test checks the handle-pinning assumption; I could not run it locally, so Windows CI is its first run. All of these except the different-file test fail on a9463b7 and pass now. The different-file test covers a guard that already existed, so it can only fail when that guard is removed.
  • Full suite on macOS: 16804 passed, 619 skipped and 1 failed. The failure is test_version_execution.py::TestWindowsUvToolEnvironment::test_non_windows_uv_tool_dry_run_still_previews_installer, which happens only on my machine and is not related to this change. ruff check is clean.

Posted on behalf of @echarrod by Claude Code (model: claude-opus-5-5), running autonomously; agent-written, no line-by-line human review.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

echarrod and others added 2 commits November 18, 2026 11:08
Rollback now holds a read-only descriptor on each file it may restore in
place (outside Windows, up to a quarter of the open-file limit), so an
original the operation unlinked cannot lend its inode to a replacement that
the identity check would then accept. A file without a pin, or whose pinned
inode has no name left, is recreated from its snapshot as before. Windows
files are not pinned, since an open file there cannot be replaced.

The Linux POSIX access ACL is now restored with the user extended
attributes, and BSD file flags that differ after an in-place restore are
reported as incomplete with the snapshots retained, as Python has no
descriptor form of chflags(). The shared CLI and API references describe
the in-place, recreated and partially restored outcomes.

Assisted-by: Claude Code (model: claude-opus-5-5, autonomous)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015w5Wio5NK7XbSPGaMEKqVe
A Linux or macOS file past the pin limit is still recreated rather than
restored in place, but when it is still the snapshotted inode at rollback,
so was probably rewritten in place, it is now reported with the recovery
snapshots retained rather than silently losing its owner and links. A file
the operation is removing is neither pinned nor restored in place.

The access ACL is restored after the user attributes, as a read-only ACL
would refuse them, archive and compression flags the filesystem changes on
a write are no longer compared, the incomplete-restore warning names
ownership, links and file flags, and closing one pin can no longer leave
the others open.

Assisted-by: Claude Code (model: claude-opus-5-5, autonomous)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015w5Wio5NK7XbSPGaMEKqVe
Copilot AI adaptive review requested due to automatic review settings November 18, 2026 11:24
@echarrod

Copy link
Copy Markdown
Contributor Author

@mnriem I've addressed the five Copilot threads from the last round in 108d3d0 and 60be027.

  • Rollback now keeps a read-only descriptor open on each file it may restore in place, so an original that was unlinked cannot lend its inode to a replacement. It does this outside Windows, for up to a quarter of the open-file limit and at most 1024 files. A file without a pin, or whose pinned inode has no name left, is recreated from its snapshot. If such a file still has its original inode, it was probably rewritten in place, so it is reported with the recovery snapshots retained. A file the operation is removing is never restored in place.
  • The Linux POSIX access ACL is now captured and restored along with the user.* attributes, and an ACL the operation introduced is removed. Both changes go through the verified descriptor, and the ACL is applied after the user attributes, since a read-only ACL would refuse them.
  • BSD file flags that differ after an in-place restore are reported as an incomplete restore, and the snapshots are kept. Flags the filesystem changes on a write (UF_COMPRESSED on macOS, UF_ARCHIVE on FreeBSD and DragonFly) are ignored. The warning now names ownership, links and file flags.
  • The timestamp-refusal test adds its wrapper to os.supports_fd, so it reaches the PermissionError handler. It fails with that handler removed.
  • The shared CLI and API references, and the _journalled docstring, now describe the in-place, recreated and partly restored outcomes.

Testing:

  • New tests: inode reuse without a link to the original, a file observed as removed and then rewritten, unpinned files that were rewritten or replaced, pin release on failure and on commit, a copy2 from an older template, access ACLs (introduced and changed), ACL ordering and BSD flags.
  • Linux (Debian container, ext4 volume, non-root): all of them run, and tests/specify_cli/integrations has 1316 passed and 3 skipped. The inode-reuse, ACL and removal tests fail on c3f7e18. The ordering test fails only when the ACL-last ordering is removed, and lists the ACL first because ext4 always lists user.* first.
  • macOS: the flag tests fail on c3f7e18. The full suite has 16813 passed, 622 skipped and 1 failed. The failure is the same machine-specific test_version_execution.py::TestWindowsUvToolEnvironment::test_non_windows_uv_tool_dry_run_still_previews_installer as before, and is not related to this change.
  • Windows: nothing has been run locally.
  • ruff check src tests (0.15.0) is clean.

Posted on behalf of @echarrod by Claude Code (model: claude-opus-5-5), running autonomously; agent-written, no line-by-line human review.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

…firmed restores

An original file is now pinned before it is snapshotted, and the snapshot is
read through the pin, so a file unlinked and recreated in the gap cannot reuse
the inode and be taken for the original.

After restoring a file in place, rollback rechecks that the path still names
the restored file; if it was removed or replaced during the copy, the restore
is reported incomplete and the recovery snapshot kept, and the concurrent
replacement is left alone.

A verified original that cannot be opened for writing and is recreated is now
reported with its snapshot retained, as an unpinned one already was, since
recreation loses its owner, links and access-control list.

Adds tests for the modification time restored on success, the pin taken before
the snapshot, a path replaced during restore, and snapshot retention when the
open is refused.

Assisted-by: Claude Code (model: claude-sonnet-5-5, autonomous)
Copilot AI adaptive review requested due to automatic review settings November 18, 2026 14:44
@echarrod

Copy link
Copy Markdown
Contributor Author

Review round summary, commit 452fb85. Four Copilot review threads addressed:

  • _lifecycle.py pin ordering: the original file is now pinned before it is snapshotted, and the snapshot is read through that descriptor, so a file replaced in the gap cannot reuse the inode and be mistaken for the original. Test: test_builtin_journal_pins_a_file_before_snapshotting_it.
  • Path replaced during an in-place restore: after the copy, rollback checks that the path still names the restored file. If not, the restore is reported incomplete and the recovery snapshot is kept; the replacement is not overwritten. Test: test_builtin_journal_keeps_the_snapshot_when_the_path_is_replaced_during_restore.
  • Verified original that cannot be opened for writing: it is recreated as before, but is now reported with its snapshot retained, like an unpinned one. test_builtin_journal_recreates_and_reports_a_file_it_cannot_open_in_place covers the warning and retention.
  • Modification-time test: test_builtin_journal_restores_the_original_modification_time asserts the saved st_mtime_ns, the same inode, and no incomplete-restore warning.

Verification: uv run --extra test pytest tests gave 11738 passed, 291 skipped, 1 failed. The failure, test_version_execution.py::TestWindowsUvToolEnvironment::test_non_windows_uv_tool_dry_run_still_previews_installer, compares wrapped output that includes the long local worktree path; it is not touched by this change.


Posted on behalf of @echarrod by Claude Code (model: claude-sonnet-5-5), running autonomously; agent-written, no line-by-line human review.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Critical rollback defects can restore stale data and leave original contents or permissions unrecovered after path replacement, while warnings may incorrectly claim that contents were restored.

2 open issues
4 resolved since last review

Review effort: Max (adaptive)

@mnriem

mnriem commented Nov 18, 2026

Copy link
Copy Markdown
Collaborator

Please address Copilot feedback

echarrod and others added 2 commits November 19, 2026 10:33
… ACL-order test

Patching os.listxattr also reordered shutil.copystat() when the journal
snapshots the file. Python before 3.13 raises EACCES there once a
read-only ACL is copied ahead of the user attributes, so the test failed
on Python 3.11 before rollback ran. The test now reorders only what
_restorable_xattrs() returns, which is what _restore_xattrs() sorts.

Assisted-by: Claude Code (model: claude-opus-5-5, autonomous)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015w5WioAe4U3H3zhK3uAy6r
…removed parent

A directory snapshotted after one of its journalled files was rewritten
and then replaced read that file through its pin, so the snapshot held
the old inode's bytes rather than the replacement. Rollback recreated
the directory with those stale bytes, then reported the file as a
conflict and left its original unrestored. A pin is now read only for
the file's own first snapshot.

The warning for an incomplete restore no longer says the contents came
back: a path replaced while its bytes are restored still holds the
replacement. It now says the file could not be fully restored, and the
rollback docs say the same.

Assisted-by: Claude Code (model: claude-opus-5-5, autonomous)
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015w5WioAe4U3H3zhK3uAy6r
Copilot AI adaptive review requested due to automatic review settings November 19, 2026 11:00
@echarrod

Copy link
Copy Markdown
Contributor Author

Review round summary for @mnriem. Head is now abcd6641. It addresses both Copilot threads from 2026-11-18 and fixes the failing pytest job.

  • Failing test (test_builtin_journal_restores_user_attributes_before_a_read_only_access_acl, 0d612332): the test patched os.listxattr globally. That also reordered shutil.copystat() when the journal snapshots the file. Python before 3.13 raises EACCES in shutil._copyxattr once a read-only ACL has been copied ahead of the user attributes, so the test failed on ubuntu/3.11 before rollback ran. Python 3.13 and later ignore EACCES there. The test now reorders only the journal's own reads (_restorable_xattrs), which is what _restore_xattrs() sorts. It still fails if that ACL-last sort is removed. The snapshot's copystat() is unchanged from main.
  • Stale parent snapshot (_lifecycle.py:261, abcd6641): a pin is now read only for the file's own first snapshot. A later snapshot of a directory that holds the file reads it from its path. New regression: test_builtin_journal_snapshots_a_replaced_file_from_its_path_when_its_parent_is_removed rewrites a file, replaces it, then removes its parent. It fails on 452fb853, where the file is left with the rewritten bytes and reported as a conflict. It passes now.
  • Warning wording (_lifecycle.py:834, abcd6641): an incomplete restore now reads "Could not fully restore ...: their contents, ownership, links, permissions, timestamps, extended attributes or file flags may differ from the originals". The rollback paragraph in docs/integrations/cli.md says the same. I reworded the warning rather than reporting path swaps as conflicts, so behavior is unchanged. The replaced-during-restore test now checks the wording.

Tests:

  • Linux, Python 3.11 and 3.14, as a non-root user in a container: test_lifecycle.py and test_command_upgrade.py give 102 passed and 3 skipped (Windows and BSD-flag cases). The ACL tests ran rather than skipped.
  • Linux, Python 3.11, tests/specify_cli and tests/integrations with xdist: 10692 passed and 51 skipped. Two runs each had one or two subprocess tests fail, and the failing tests differed between runs: test_stdio.py::test_real_stdio_server_initializes_discovers_and_calls_first_class_tools and test_installed_adapters.py::test_round3_extension_event_refresh_loads_adapter_in_fresh_process[disable]. Both modules pass when run on their own (689 passed).
  • macOS, the same two lifecycle modules: 99 passed, 6 skipped.
  • ruff on the changed files reports no new findings.

Posted on behalf of @echarrod by Claude Code (model: claude-opus-5-5), running autonomously; agent-written, no line-by-line human review.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Human assessment is needed for unpublished concerns about rollback-test snapshot leaks and documentation that overstates in-place restoration guarantees.

No open issues 🎉

2 resolved since last review

Review effort: Max (adaptive)

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The recovery-incomplete test leaves a snapshot in the system temporary directory on every run without cleaning it up.

1 open issue

Review effort: Max (adaptive)

@mnriem

mnriem commented Nov 19, 2026

Copy link
Copy Markdown
Collaborator

Please address Copilot feedback

…riptor test

test_builtin_journal_never_writes_a_different_file_open_returned marks
recovery incomplete, so rollback keeps its snapshot outside tmp_path and
every run left a directory in the system temporary directory. The test
now records the backup and removes it, as the other incomplete-recovery
tests do.

Assisted-by: Claude Code (model: claude-sonnet-5-5, autonomous)
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_015w5WioL9JfLpDRHzdjgcDf
Copilot AI adaptive review requested due to automatic review settings November 19, 2026 17:18
@echarrod

Copy link
Copy Markdown
Contributor Author

Review round summary for @mnriem. Head is now e4fb9d9d. It addresses the Copilot thread from 2026-11-19 on test_lifecycle.py.

  • Leaked rollback backup (e4fb9d9d): test_builtin_journal_never_writes_a_different_file_open_returned marks recovery incomplete, so rollback keeps its snapshot outside tmp_path. The test now records the backup with _record_rollback_backups() and removes it with shutil.rmtree(), as the other incomplete-recovery tests do. Only the test changed.

Tests: macOS, tests/specify_cli/integrations/test_lifecycle.py gives 30 passed and 6 skipped. The system temporary directory holds no new speckit-integration-rollback- directories after the run.


Posted on behalf of @echarrod by Claude Code (model: claude-sonnet-5-5), running autonomously; agent-written, no line-by-line human review.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Human assessment is needed to determine the approval impact of an unpublished nit about misleading documentation of shared file offsets.

No open issues 🎉

1 resolved since last review

Review effort: Max (adaptive)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

triage-must-have Verdict: high-value, important work for Spec Kit — do first

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants